Dynamic Adaptive Streaming using Index-Based Learning Algorithms

We provide a unified framework using which we design scalable dynamic adaptive video streaming algorithms based on index based policies (dubbed DASIP Fig. 2) to maximize the Quality of Experience (QoE) provided to clients using video streaming services. Due to the distributed nature of our algorithm DAS-IP, it can be easily implemented in lieu of popular existing Dynamic Adaptive Streaming over HTTP (DASH) algorithm which is used by various Cloud based video streaming services, Content Delivery Networks (CDNs), Cache networks, wireless networks, vehicular networks etc. We begin by considering the simplest set-up of a onehop wireless network in which an Access Point (AP) transmits video packets to multiple clients over a shared unreliable channel. The video file meant for each client has been fragmented into several packets, and the server maintains multiple copies (each of different quality) of the same video file. Clients maintain individual packet buffers in order to mitigate the effect of uncertainty on video iterruption. Streaming experience, or the Quality of Experience (QoE) of a client depends on several factors: i) starvation/outage probability, i.e., average time duration for which the client does not play video because the buffer is empty, ii) average video quality, iii) average number of starvation periods, iv) temporal variations in video quality etc. We pose the problem of making dynamic streaming decisions in order to maximize the total QoE as a Constrained Markov Decision Process (CMDP). A consideration of the associated dual MDP suggests us that the problem is vastly simplified if the AP is allowed to charge a price per unit bandwidth usage from the clients. More concretely, a “client-by-client” QoE optimization leads to the networkwide QoE maximization, and thus provides us a decentralized streaming algorithm. This enables the clients to themselves decide the optimal streaming choices in each time-slot, and yields us a much desired client-level adaptation algorithm. The optimal policy has an appealing simple threshold structure, in which the decision to choose the video-quality and power-level of transmission depends solely on the buffer-level. In case the clients are unaware of their (possibly random and timevarying) system parmeters, we develop algorithms that learn the indices while utilizing the strucure of the optimal decentralized policy. The decentralized nature of optimal policy implies that the DAS-IP has a much “smaller” policy space to explore from, and hence converges fast.